The smallest farms, those making less than $10,000 annually, experienced the biggest decline as some owners retired and others were purchased by larger operations, the USDA said.
Small farms declined to 1.18 million farms, down 1.6 percent from 2003. Large operations, those making more than $500,000 annually, increased 4.7 percent to 76,740 farms.
The average US farm had 443 acres in 2004, an increase of two acres from the previous year.
Hog operations experienced the largest decline among livestock producers, falling 6 percent from 2003 to 69,420. Cattle operations totalled 989,460, down 2 percent.